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Abstract 


Problems 


Problems  1)  Poor  quality  of  commercial  wireless  Internet  connectivity  We  frequently 
faced  poor  Internet  connection  with  a  commercial  wireless  Internet  service  provider, 
which  frustrated  users.  This  is  anticipated  to  improve  as  service  improves  regionally  and 
globally.  2)  Little  evidence  for  clinical  decision  making  We  decided  to  develop  a  decision 
support  system  for  crush  injury  because  little  epidemiological  study  is  available  for  other 
disaster  unique  medical  diagnoses.  3)  Narrow  clinical  application  Considering  the  clinical 
application  of  such  decision  support  systems,  more  benefit  will  be  provided  by  decision 
support  tools  for  more  common  diseases  and  injuries.  Future  direction  1)  Improve 
connectivity  A  more  stable  network  connection  is  required  for  clinical  use  of  the  system. 
Stand-alone  applications  that  can  store  the  data  in  a  handheld  device  are  necessary  to 
develop.  2)  Security  and  confidentiality  Security  and  confidentiality  such  as  encryption, 
IP  filtering  and  authentiflcation,  should  be  considered.  These  issues  are  for  the  HIPAA 
(Health  Insurance  Portability  and  Accountability  Act)  regulation.  3)  Wider  clinical 
application  Decision  support  systems  (DSS)  that  will  cover  much  wider  clinical 
applications  will  be  beneficial.  Clinical  DSS  for  further  predictive  model  development  is 
planned.  4)  User  interface  Development  of  optimal  user  interface  for  the  decision  support 
system  with  users  is  necessary. 


Deliverables 


1)  Predictive  model  development  We  developed  a  predictive  model  for  patients  with 
crush  injury  using  existing  dataset  from  the  Hanshin-Awaji  Earthquake.  We  studied  12 
possible  predictive  factors,  which  is  available  at  the  field  examination:  age  and  gender, 
respiratory  rate,  systolic  blood  pressure,  pulse  rate  and  availability  of  urine  specimen 


T 


with  evaluation  of  urine  color  /  rescue  time:  hours  from  earthquake  impact  to  patient 
extrication  and  time  from  extrication  to  initial  patient  examination,  volume  of 
intravenous  fluid  during  the  first  three  days  following  the  earthquake,  and  Injured 
anatomic  sites.  A  logistic  regression  model  was  used  to  build  a  predictive  model  to 
estimate  deleterious  outcomes  defined  as  hemodialysis  and/or  death  (severe  and  fatal 
crush  syndrome).  Total  of  330  patients  data  were  split  into  two  parts:  training  data  set 
(220  cases)  and  test  data  set  (110).  The  training  data  set  was  used  to  construct  logistic 
regression  model  and  the  test  data  set  was  used  to  test  a  validity  of  the  model.  The  three 
prognostic  factors,  odds  ratio  and  95%  confidence  intervals  in  the  final  logistic 
regression  model  are  listed  in  table  1. 2)  Web-based  database  system  development  Figure 
1  shows  a  structure  of  the  web-based  database  system.  Oracle  8i  application  program  has 
been  used  to  develop  the  web-based  database  system.  As  the  illustration  shows,  a  table  in 
a  relational  database  is  organized  in  rows  and  columns.  Each  column,  called  a  field, 
represents  a  specific  type  of  data  stored  in  the  table.  For  example,  in  the  PATIENT  table, 
columns  include  the  patient’s  ID,  last  name,  and  sex.  Each  row  called  a  record  represents 
a  set  of  related  data  about  a  single  entity,  such  as  a  person.  Many  rows  make  up  a  table. 
For  example,  each  row  in  the  PATIENT  table  represents  a  patient’s  demographic 
information.  The  patient  ID  identifies  each  patient  or  contact  in  the  PATIENT  table.  The 
ID  is  used  as  a  key  in  many  tables,  which  means  that  this  ID  is  used  to  refer  any 
information  belong  to  each  patient  (e.g.  age,  gender)  from  other  tables.  Since  each  patient 
might  have  several  data  entry  points,  ID  in  the  OBS  table  named  as  OBSID,  is  used  as 
unique  identifier  for  the  each  record  stored  in  the  database  system.  This  OBSID  is  used 
to  identify  each  record  stored  in  the  database,  which  is  automatically  generated.  Care 
providers,  however,  are  not  required  to  input  the  OBSID  to  retrieve  the  stored  data. 

They  can  input  patient  ID,  first  name,  or  last  name  to  identify  the  patient.  Then,  the 
database  system  would  return  a  list  of  possible  patients,  so  that  care  providers  can 
identify  the  patient.  ID  and  password  is  required  to  access  any  data  stored  in  the 
database  system.  3)  Decision  support  system  development  Specification  for  the  decision 
support  system  Server:  IRIX64  Operating  system:  IRIX  6.5  version  Network:  Internet 
Language:  Java 

Software  on  server  JDK:  1.3  version  Apache  Jakarta-  Tomcat:  3x  version  JDBC  driver: 
1.2  version 

Client  Browser:  PC:  IE  /Netscape  browser  Handheld:  Palmscape 

Handheld  Hardware:  Palm  Vx  and  Visor  Prism  Operating  system:  Palm  OS  3.5  version 

Network:  commercial  wireless  ISP  (Omnisky  Inc.)  A  JSP  (Java  Servlet  Page)  program 
running  on  a  server  creates  web  pages,  which  can  be  accessed  by  Web  browser  or  Palm 
browser.  The  program  interacts  with  the  user,  gets  patient’s  demographic  data  directly 
from  database  and  then  collects  and  inserts  the  patient’s  observation  data  into  the 
database.  The  system  calculates  the  possibility  of  Crush  Injury  Syndrome  and  provides  a 
suggestion  of  how  to  handle  the  patient.  Decision  support  system  for  handheld  In  the 
initial  screen  of  the  decision  support  system,  care  providers  must  input  their  user  ID  and 
password  in  order  to  enter  the  decision  support  system.  In  the  next  step,  care  providers 
are  asked  to  input  patient  demographics  information.  The  web-based  database  system 
automatically  generates  a  patient  ID.  Care  providers  can  put  the  ID  on  the  triage  tag  for 
future  information  retrieval  (Figure  2).  Then,  care  providers  are  asked  to  input 
important  physiologic  information  of  the  patient  including  vital  signs  and  important 
predictive  factors  (Figure  3).  All  the  data  input  here  is  stored  in  the  database  for  future 
retrieval.  After  submitting  these  data,  the  server  computer  calculates  a  possibility  of 
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deleterious  outcomes  based  on  the  data  input  in  the  handheld,  and  sends  them  to  care 
providers.  Figure  4  shows  final  decision  support  screen.  Estimated  probability  of 
deleterious  outcomes  and  95  %  confidence  intervals  are  visualized  on  the  screen. 
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Final  Results 

1)  Field  test  The  configured  PDA  and  wireless  connection  to  the  server  were  user-tested 
by  six  Emergency  physicians,  nurses  and  medics  from  Tripler  Army  Medical  Center.  The 
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individuals  were  asked  to  input  fictitious  patient  data  and  send  it  wirelessly  to  the  server. 
The  events  observed  consisted  of  1)  ease  of  use,  2)  rapidity  of  use,  3)  any  user  learning 
that  was  required,  4)  preferences  in  use.  The  test  took  place  outdoors  for  maximum 
connectivity.  Two  units  were  unable  to  establish  connectivity  rapidly.  The  users  found 
the  PDA  to  be  easy  to  use  in  its  configuration  and  fields  for  input.  Rapidity  of  use 
depended  on  an  individual’s  prior  experience  with  a  PDA.  Those  familiar  with  the  PDA 
were  rapid  in  input  of  patient  data  and  sending  it  to  the  server.  They  were  also  pleased  to 
receive  within  seconds,  a  calculated  decision  from  the  server.  Those  not  familiar  with  the 
PDA  were  able  to  quickly  learn  (within  5  minutes)  the  process  (although  the  graffiti 
feature  distracted  them  and  they  preferred  the  keyboard  to  avoid  mistakes).  Most 
individuals  preferred  the  keyboard  for  inputting  patient  data.  The  overall  comments 
about  the  system  were  very  positive  and  the  individuals  agreed  that  this  system  could 
make  important  contributions  to  emergency  healthcare.  2)  Comparison  between  expert 
opinion  and  decision  support  system  for  decision  making  with  cross-sectional  data 
Expert  opinion  Two  medical  doctors,  two  nurses  and  two  medics  participated  to  this 
trial.  Following  information  of  these  patients  was  distributed  to  each  responder,  which 
was  derived  from  actual  patients  with  crush  injury  at  the  Hanshin-Awaji  Earthquake: 
age,  gender,  injury  site  (head,  chest  abdomen,  right  arm,  left  arm,  right  leg  and  left  leg), 
time  until  rescued,  respiratory  rate,  systolic  blood  pressure,  pulse  rate,  availability  of 
urine  and  urine  color  if  available.  Each  responder  was  asked  to  roughly  estimate  the 
probability  of  having  a  deleterious  outcome  defined  as  receiving  hemodialysis  or  death. 
One  hundred  twenty  patient  records  were  selected  because  they  had  almost  complete 
information.  Each  responder  was  asked  to  estimate  a  probability  of  deleterious  outcome 
(defined  above)  for  each  patient.  Validation  of  the  prognostic  model  The  remaining  110 
cases  (test  dataset)  were  used  to  validate  the  predictive  model.  The  missing  values  in  this 
data  set  were  inputted  to  correctly  evaluate  the  usefulness  of  this  predictive  model  in  a 
real-world  situation.  Evaluation  of  diagnostic  accuracy  All  raw  probabilities  derived 
from  both  the  decision  support  system  and  experts  were  used  to  generate  receiver- 
operating  (ROC)  curve  to  evaluate  performance.  The  ROC  curve  represents  the 
relationship  between  sensitivity  and  specificity,  by  plotting  the  true-positive  rate 
(sensitivity)  against  the  false-positive  rate  (1-  specificity)  as  the  cutoff  level  of  the  model 
varies.  The  area  under  the  ROC  curve  (AUC)  is  based  on  a  non-parametric  statistical 
sign  test  to  compare  the  probability  of  events  between  pairs  of  patients  who  have  the 
event  and  those  who  do  not.  AUC  may  be  interpreted  as  the  probability  that  given  any 
two  subjects,  one  who  dies  and  one  who  survives,  the  model  would  assign  a  higher 
probability  of  death  to  the  one  who  dies.  It  is  a  measure  of  overall  classification 
performance  of  a  diagnostic  test  or  prognostic  model.  Then,  all  raw  probabilities  were 
converted  into  binary  categories.  We  used  50%  of  probability  as  a  threshold  of  possible 
good  and  bad  outcomes.  This  means  that  any  estimated  probabilities  less  than  50%  were 
classified  as  a  “good  outcome”  and  that  greater  than  or  equal  50%  were  classified  as 
“deleterious  outcome”.  These  results  were  compared  to  the  actual  outcome  data  in  each 
responder  including  the  computer-based  decision  support  system  to  calculate  sensitivity 
and  specificity.  2)  Results  ROC  curve  and  AUC  Figure  5  shows  the  ROC  curve  for 
responder  A  to  F  and  the  decision  support  system.  Table  2  shows  the  AUC  of  each  ROC 
curve.  The  decision  support  system  showed  the  best  AUC,  which  is  statistically 
significantly  better  than  coin-  flipping  decision  making  with  cross  sectional  data. 
Sensitivity  and  Specificity  Table  3  shows  the  overall  accuracy,  sensitivity  and  specificity 
of  the  decision  support  system  and  experts’  decision  making.  As  the  AUC  shows,  the 
overall  accuracy  of  the  decision  support  system  is  the  best  of  all.  Sensitivity  and 
specificity  of  expert  decision  making,  however,  shows  some  interesting  aspects  of  medical 
decision  making  in  a  triage  situation.  Specificity  by  expert  B,  C  and  E  showed  extremely 
high  values  compared  with  the  decision  support  system.  It  means  that  these  experts  tried 
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to  identify  very  severe  patients  who  required  immediate  rescue  activities  (aggressive 
triage).  In  contrast,  expert  D  seemed  to  try  to  increase  sensitivity.  This  means  that  the 
expert  tried  to  identify  as  many  patients  as  possible  to  avoid  false  negatives  (conservative 
triage).  Expert  A  and  F  showed  similar  attitudes  as  the  decision  support  system  (neutral 
triage^  This  kind  of  decision  totally  depends  on  purpose,  situation  and  location  of  triage 
as  well  as  resource  availability.  Naturally,  the  computer  cannot  make  such  a  decision 
without  any  input  regarding  this  information.  Therefore,  these  results  showed  an 
important  difference  between  computer-based  decision  making  and  expert  decision 
making,  and  roles  of  computer-based  decision  support  in  triage.  Computers  may  be  able 
to  provide  good  results  if  the  care  provider  can  give  the  system  crucial  information,  such 
as  purpose,  situation  and  location  of  triage  as  well  as  resource  availability,  (which  the 
computer  cannot  determine).  The  results  also  showed  the  difficulty  of  doing  triage  with 
cross-sectional  data  entry  (care  providers  usually  make  a  decision  using  time  sequential 
information).  Therefore,  this  is  an  important  role  of  a  computer-based  decision  support 
system. 


Projected  Costs 


Q  Current  commercial  wireless  Internet  service  is  not  enough  for  this  purpose.  We  need 
to  use  alternative  technology  to  solve  this  important  limitation,  o  We  have  identified 
concrete  solutions  for  this  issue  during  this  project  and  plan  to  implement  these  in  future 
projects.  Therefore,  we  cannot  calculate  the  cost  at  this  time. 


Comments 


o  We  realized  that  technologies  have  been  improving  rapidly  during  this  project.  Some 
useful  technologies  that  were  not  available  when  we  submitted  the  proposal  are  now 
extensively  applied,  o  Considering  this  issue,  we  were  able  to  try  several  advanced 
technologies  in  telemedicine  in  order  to  solve  current  problems  in  disaster  medicine. 


TATRC  Scientific  Review 


TATRC  Acquisition  Review 


Supporting  Graphs/Cliarts 


See  Attached 


6 


10/9/2001  12:07  P 


Table  3.  Sensitivity  and  specificity 


Decision  Support  System 
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